Cross-Validation Estimates IMSE
نویسندگان
چکیده
Integrated Mean Squared Error (IMSE) is a version of the usual mean squared error criterion, averaged over all possible training sets of a given size. If it could be observed, it could be used to determine optimal network complexity or optimal data subsets for efficient training. We show that two common methods of cross-validating average squared error deliver unbiased estimates of IMSE, converging to IMSE with probability one. These estimates thus make possible approximate IMSE-based choice of network complexity. We also show that two variants of cross validation measure provide unbiased IMSE-based estimates potentially useful for selecting optimal data subsets.
منابع مشابه
Spectral approximation of the IMSE criterion for optimal designs in kernel-based interpolation models
We address the problem of computing IMSE-optimal designs for random field interpolation models. A spectral representation of the IMSE criterion is obtained from the eigendecomposition of the integral operator defined by the covariance kernel of the random field and integration measure considered. The IMSE can then be approximated by spectral truncation and bounds on the error induced by this tr...
متن کاملApproximation of IMSE-optimal Designs via Quadrature Rules and Spectral Decomposition
We address the problem of computing IMSE (Integrated Mean-Squared Error) optimal designs for random fields interpolation with known mean and covariance. We both consider the IMSE and truncated-IMSE (approximation of the IMSE by spectral truncation). We assume that the MSE is integrated through a discrete measure and restrict the design space to the support of the considered measure. The IMSE an...
متن کاملApproximation of IMSE-optimal designs via quadrature rule and spectral decomposition
We address the problem of computing IMSE (Integrated Mean Square Error) optimal designs for random fields interpolation with known mean and covariance. We assume that the MSE is integrated through a discrete measure and restrict the design space to the support of the measure considered. For such quadrature designs, the computational cost of IMSE evaluations can be significantly reduced by consi...
متن کاملAutomatic Selection of Reliability Estimates for Individual Regression Predictions Using Meta-Learning and Internal Cross-Validation
In machine learning and its risk-sensitive applications (e.g. medicine, engineering, business), the reliability estimates for individual predictions provide more information about the individual prediction error than the average accuracy of predictive model (e.g. relative mean squared error). Furthermore, they enable the users to distinguish between more and less reliable predictions. The empir...
متن کاملAlternate Test of LNAs Through Ensemble Learning of On-Chip Digital Envelope Signatures
This paper presents a novel and low-cost methodology for testing embedded Low Noise Amplifiers (LNAs). It is based on the detection and analysis of the response envelope of the Device Under Test (DUT) to a two-tone input signal. The envelope signal is processed to obtain a digital signature sensitive to key specifications of the DUT. An optimized regression model based on ensemble learning is u...
متن کامل